3D Hand Tracking with Stereo Cameras

We propose a method for extracting 3D position and posture of hands with stereo cameras. The main contribution of our research is that our method enables to track the position and posture in mobile environments by using stereo cameras. The hand is often observed with many occluded regions, since the hand has many joints. In previous methods, many cameras are required to extract the position and posture. In this research, the position and posture of the hand are estimated from 3D alignment of feature points on the surface of the hand. Stereo cameras enable to extract 3D alignment of the feature points. The position of each feature point is identified by matching with the model. Even if the surface is deformed, the local alignment of the feature points is not drastically changed. Each feature point is identified with pose-invariant feature that is a combination of the feature of the points and the relative position to the neighboring feature points. In this research, the feature points are given by wearing gloves with a known pattern. Experimental results show that the position of the feature points on gloves can be tracked in stereo images, which is not dependent on the posture of the hand.